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METHOD AND APPARATUS FOR CORRELATING 
SYSTEM RESOURCES TO A PARTICULAR LINE CORD 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to improving the reliability of high reliability 
systems. More specifically, the present invention relates to identifying connections 
of uninterruptible power supplies to system resources. 

Description of the Related Art 

[0002] High reliability is often required in mission critical applications such as 
those that occur in medical, financial, communication, and military systems. Such 
systems can become very large and complex, involving numerous sub-systems that 
are integrated together by complex interconnections. For example, computer 
systems that process financial data can involve racks of hardware and can include 
hundreds of sub-systems, each with its own processors and power supplies. 
Another example is an Internet communication system that is comprised of 
numerous distributed servers. Such systems usually include a service processor that 
controls and integrates the individual sub-systems together under the direction of 
operating software. 

[0003] Reliability can be so important that some systems have sub-systems with 
redundant power supplies that are connected to different power lines which are fed 
by different circuit-breakers. Other applications, e.g., communication servers, 
distribute user demands over distributed networks that are powered by different 
power lines that pass through different circuit breakers. Some applications are so 
critical that the different circuit-breakers are themselves powered by different power 
companies. When reliability is important the use of uninterruptible power supplies 
(UPS) is common. A UPS provides backup power in the event of an electrical 
outage or other power line disturbance. A UPS usually includes a battery system 
that powers an inverter that supplies operating power to the protected system or 
sub-system. The UPS can either power the system or sub-system only when a 
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failure occurs or it can power the system or sub-system continuously with the input 
AC power acting as a re-charger for the batteries. 

[0004] Some dual-power line systems use only one UPS that is fed from one AC 
power line. In such cases the other AC power line is directly connected to the other 
power line. In other dual-power line systems, each AC power line connects to a 
different UPS. Either way, to maintain reliable operation it is important to ensure 
that each of the redundant power supplies is powered by a different AC power 
source. Otherwise, the purpose of having dual-power lines is defeated. In 
distributed system many different sub-systems are often connected to the same 
UPS. If that UPS fails those sub-systems can be brought down. 

[0005] What the foregoing systems have in common is a need for high reliability 
and the use of UPS systems to assist that reliability. However, no matter how 
reliable any power source is, it can fail. For example, if input power is removed from 
a UPS, eventually the UPS battery will fail. If dual-lines are feed from different 
circuit breakers powered by the same power company, the power company can fail. 
If different power companies are used, an electrical grid malfunction can shut-down 
both power companies. 

[0006] While input power cannot be guaranteed, it is possible to provide 
controlled shut-down of systems to prevent, reduce, or mitigate problems. To 
improve reliability and/or to assist controlled shut-down it can be very useful to know 
what system resource is being powered by any particular USP system. Then, when 
that UPS has a problem or signals that a power failure may occur the service 
processor can perform a controlled shut-down of impacted resources. 
Unfortunately, determining what system resource is connected to a particular UPS 
becomes increasingly difficult as the number of UPS and system resources increase 
and as the system become more and more distributed. Complicating the problem is 
the desirability of determining which system resource is ultimately connected to a 
particular AC power line in a dual-line system. Further complicating the problem is 
providing sufficient time for a system to perform a controlled shut-down of system 
resource prior to power failure. 

[0007] Therefore, techniques of identifying which system resource is connected 
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to which UPS would be useful. Information that can be used to ensure that 
redundant power supplies are operatively connected to different UPS devices and to 
different AC power lines also would be beneficial. Also beneficial would be 
techniques of determining which system resources are powered by which UPS and 
which AC input line. Also beneficial would be a method of ensuring that sufficient 
notice is given before a particular system resource fails to provide for a controlled 
shut-down. 

SUMMARY OF THE INVENTION 

[0008] The principles of the present invention provide for methods and 
apparatuses that determine which system resources are connected to which UPS, 
and possibly which UPS is connected to which AC power line. The determined 
information can be used to ensure that redundant power supplies are operatively 
connected to different UPS systems and to different AC power lines. That 
information can also be used to determine which system resources are powered by 
which UPS, and possibly which AC input line. This enables a controlled shutdown of 
system resources, such as by shifting operating loads, can be provided when a UPS 
signals that a power failure is possible. 

[0009] In one embodiment of the present invention, a user prepares a 
configuration file that describes the interconnections of the UPS system with the 
system resources. The operating system can check the configuration file for 
redundancy errors to ensure that redundant power supplies are not connected to the 
same UPS and/or AC power line. The operating system could also check to 
determine which system resource is connected to which UPS. Then, when a UPS 
signals that a power failure may occur the system can reference the configuration 
file to determine how to perform a controlled shutdown of system resources to 
mitigate damage. 

[0010] In another embodiment, a UPS sends identifying information, such as an 
IP address or serial number, on an input AC power line. The identifying information 
is subsequently detected by the sub-system or service processor, which then checks 
to ensure that redundant power supplies are not connected to the same UPS and/or 
to enable a controlled shutdown of system resources. 
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[0011] In another embodiment of the present invention, a service processor 
includes power microcode that controls the output voltage of each UPS. That power 
microcode then directs a particular UPS to raise or lower its output voltage. The 
sub-systems include voltage triggers that signal when UPS power is out-of-range. 
When the particular UPS achieves an out-of-range condition the sub-system or 
service processor detects the out-of-range signals and identifies the sub- 
system/UPS interconnection topology. Checks can then be made to ensure that 
redundant power supplies are not both out-of-range, and thus are both not 
connected to the particular UPS, or a configure table is generated in which specific 
UPS systems are associated with specific system resources. Then, when a UPS 
signals a potential power failure the system can take steps to provide a controlled 
shutdown of system resources that are associated with the failing UPS. 

[0012] In another embodiment, a service processor includes power microcode 
that directs a particular UPS to turn off its output. When the particular UPS turns off 
the power to a sub-system the service processor detects the OFF condition and 
checks to ensure that redundant power supplies are not both OFF, and thus both 
are not connected to the particular UPS. Alternatively, a configure table is generated 
in which specific UPS systems are associated with specific system resources. 
Then, when a UPS signals a potential power failure the system can take steps to 
provide a controlled shutdown of system resources that are associated with the 
failing UPS. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0013] So that the manner in which the above recited features of the present 
invention can be understood in detail, a more particular description of the invention, 
briefly summarized above, may be had by reference to embodiments, some of which 
are illustrated in the appended drawings. It is to be noted, however, that the 
appended drawings illustrate only typical embodiments of this invention and are 
therefore not to be considered limiting of its scope, for the invention may admit to 
other equally effective embodiments. 

[0014] Figure 1 A is a schematic depiction of a high reliability system having sub- 
systems with redundant power supplies and multiple UPS devices powered from 
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different AC power lines; 

[0015] Figure 1 B is a schematic depiction of a high reliability system having 
distributed sub-systems and multiple UPS devices powered from different AC power 
lines; 

[0016] Figure 2 is a flow diagram of a first process for checking the integrity of 
the power supplies and of the UPS systems; 

[0017] Figure 3 illustrates a data table suitable for implementing the principles of 
the present invention; 

[0018] Figure 4 is a flow diagram of a second process for checking the integrity of 
the power supplies and of the UPS systems; and 

[0019] Figure 5 is a flow diagram of a third process for checking the integrity of 
the power supplies and of the UPS systems. 

[0020] To facilitate understanding, identical reference numerals have been used, 
wherever possible, to designate identical elements that are common to the figures. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0021] The principles of the present invention provide for methods and 
apparatuses that improve system reliability by identifying UPS-sub-system 
interconnections. This enables protecting the system against UPS connection 
problems. In systems that use redundant power supplies, the UPS-sub-system 
interconnections are checked to ensure that each redundant sub-system power 
supply is operatively connected to a different UPS, and possibly to a different AC 
power line. In systems having distributed sub-systems, the sub-system-UPS 
connections are identified to determine which UPS is connected to which sub- 
system. Once such information is available the system can perform a controlled 
shutdown of system resources in the event of a UPS failure. 

[0022] Figure 1 A is a schematic depiction of a first embodiment high reliability 
system 100 that is suitable for practicing the present invention. The system 100 
includes numerous subsystems, each of which includes a redundant power supply. 
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The system 100 includes a service processor 102 that controls and monitors the 
overall operation of the system 100. The service processor 102 is operated by 
software that is stored in memory 101 and that implements the overall purpose of 
the system 100. As the present invention relates to ensuring the integrity of the 
system power, part of that software, referred to herein as power micro-code, will be 
specifically discussed. 

[0023] The system 100 includes a plurality of N (an integer) sub-systems, 
illustrated by the subsystems 104, 106, 108, and 1 10, each of which includes two 
power supplies, labeled 1 04A-104B through 1 10A-1 10B. Each power supply can 
individually power its associated sub-system, e.g., the power supply 106A can 
completely power the sub-system 106. Thus, the pairs of power supplies 104A- 
104B through 1 10A-1 10B are redundant in that only one needs to operate. 

[0024] Still referring to Figure 1 A, each power supply is powered by an 
uninterruptible power supply UPS. As shown, the system 100 has two UPS 
supplies, a UPS 120 and a UPS 122. One power supply of each subsystem is 
powered by UPS 120 while the other power supply is powered by UPS 122. The 
UPS 120 receives its power from AC power line #1 while the UPS 122 receives its 
power from the UPS power line #2. Since each power supply is connected to a 
UPS, and since each UPS is connected to an AC power line, each power supply is 
operatively connected to an AC power line. This is because an AC power line 
sources operating power even though a UPS comes between the AC power line and 
a power supply. While Figure 1 shows only 2 UPS, some systems may have only 
one UPS while others may have three or more, possibly many more. An example of 
a system with more than two UPS is illustrated in Figure 3. Additionally, while 
Figure 1 shows only two AC power lines, some systems may use three or more. 
What is important is that the power supplies of a particular sub-system are ultimately 
powered from different AC power lines. The overall goal is to ensure that a failure of 
one AC power line does not cause both power supplies of a sub-system to fail (thus 
preserving their redundant status). 

[0025] In the system 1 00 the service processor 1 02 can send power microcode 
commands via bus 130 to each UPS. That code controls the operation of each 
UPS. Also, each UPS has a discrete IP address or other identifier, such as a serial 
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number, that can uniquely identify the UPS address, and the ability to send that 
identifier to the system 100. 

[0026] Figure 1 B is a schematic depiction of a second embodiment high reliability 
system 150 that is suitable for practicing the present invention. The system 150 
includes a plurality of distributed subsystems which are not all connected to the 
same UPS system or to the AC input line. As shown, the system 150 includes a 
service processor 152 that controls and monitors the overall operation of the system 
150. The service processor 152 is controlled by software that is stored in memory 
1 51 and that implements the overall purpose of the system 150. As the present 
invention relates to identifying the interconnections of the UPS devices and sub- 
systems, part of that software, referred to herein as power micro-code, will be 
specifically discussed. 

[0027] The system 150 includes a plurality of N (an integer) sub-systems that are 
illustrated by the servers (1-5), which are identified as server devices 154, 156, 158, 
160, and 162. Each server device handles communications with a plurality of users. 
The server devices 154, 156, 158, 160, and 162 connect to uninterruptible power 
supplies UPS1, UPS2, and UPS3, which are identified as UPS devices 164, 166, 
and 1 68. The UPS device 1 64 receives its input power from AC power line #1 , UPS 
device 166 receives its input power from AC power line #2, and UPS device 168 
receives its input power from AC power line #3. Since each server is connected to a 
UPS, and since each UPS is connected to an AC power line, each server is 
operatively powered by an AC power line. 

[0028] In the system 1 50 the service processor 1 52 can send power microcode 
commands via bus 170 to each UPS device that controls the operation of each UPS. 
Also, each UPS device has a discrete IP address or other identifier, such as a serial 
number, that uniquely identifies the UPS device. Each UPS device also includes the 
ability to send that identifier to the system 150. 

[0029] Figure 1 A and 1 B illustrate two useful embodiments of the present 
invention: redundant power supply systems and distributed systems, respectively. 
However, other systems also will be able to make use of the principles of the 
present invention. Those principles include determining which sub-system is 
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connected to which UPS, and possibly which UPS is connected to which AC power 
line. 

[0030] Figure 2 illustrates a first process 200 for verifying and identifying the UPS 
connections to the sub-systems or to the sub-system power supplies. Again, the 
overall purpose is to determine which UPS powers which system resource. Such 
information enables the system to ensure that each redundant subsystem power 
supply is ultimately powered by different AC power line/UPS, or to enable a 
controlled shutdown of system resources in the event of power failure. As shown in 
Figure 2, the process 200 starts at step 202 and proceeds, at step 204, with a 
manual entry of a listing of power supplies/subsystems to UPS connections and their 
connections to AC power lines. An exemplary data table 300 for such a listing when 
considering redundant power supplies is shown in Figure 3. That data table is 
stored in memory 101 (see Figure 1A) or in memory 151 (Figure 1 B). In the system 
100, after listing, at step 206, the power microcode checks the table to verify power 
integrity by ensuring that each sub-system has power supplies that eventually 
connect back to different AC power lines. If not, at step 208 the system operator is 
notified of a power integrity conflict. Then, at step 210, the method 200 stops. 

[0031] A similar method is used when determining which sub-system is powered 
by which UPS/AC power line. In that case, if a UPS signals a potential failure the 
server 152 can perform a controlled shutdown of sub-systems that might fail. For 
example, in Figure 1A, if the UPS device 166 signals that UPS 2 might fail, the 
service processor 152 can switch user communications from server 3 (server device 
158) to server 2 (server device 156) before UPS 2 (UPS device 166) actually fails. 
Thus, a tabular listing of sub-system/U PS/AC power line connections can enable a 
controlled shutdown of sub-systems in time to prevent loss of service. 

[0032] Referring now to Figure 3, the data table 300 shows sub-systems 1 and 2 
having power supplies that are driven by different UPS devices (note that Figure 3 
lists four UPS devices), and that the power supplies for sub-systems 1 and 2 
connect to different AC lines. However, while sub-system 3 has power supplies 
driven by different UPS devices, those UPS devices are powered by the same AC 
line. This represents a fault condition and a warning (illustrated by highlighting) is 
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provided to the system operator. 

[0033] While manual entry of information is useful, as a system becomes larger 
and more complex, the manual entry method becomes increasing susceptible to 
errors. Figure 4 illustrates a second process 400 for identifying which UPS is 
connected to which system resource. Again, this information can be used to ensure 
power system integrity of UPS connections with sub-system power supplies. 

[0034] As shown in Figure 4, the process 400 starts at step 402 and proceeds, at 
step 404, by having all of the UPS devices produce power. At step 406, the UPS 
devices send their individual IP addresses (or other identifier) on their power lines to 
the various power supplies (server devices) that they drive. Sending such data can 
be accomplished using RF modulated signals that are capacitive coupled to the 
power lines. At step 408, the power supplies/subsystems/service processors receive 
the IP address by stripping the IP address from the power supply lines. Again, this 
can be accomplished by capacitive de-coupling of the RF modulated signals from 
the power lines. Then, at step 410, the individual sub-system or the service 
processor identifies which UPS connects to which system resource. If appropriate, 
at optional step 412, verification is made that the IP addresses, and thus the UPS 
devices are associated with different AC power lines. This requires some prior 
knowledge about which UPS is connected to which AC power line. This knowledge 
can be physically entered into the system. Finally, at step 414, the method stops. 

[0035] If the method of sending UPS device identifiers is used with the system 
150 shown in Figure 1B, step 410 can be skipped, and the UPS-to-server 
information can be automatically stored. Then, if a UPS power failure is signaled by 
a particular UPS the service processor 152 can provide for a controlled shut-down of 
servers powered by the UPS that may fail, such as by switching communications to 
other servers. 

[0036] While the method 400 beneficially provides for automated determination of 
which power supply/server is powered by which UPS, thus enabling automated 
power integrity verification, the method requires both identification information and 
stripping of that information from the power supply input lines. While not technically 
difficult to do, the method 400 requires additional hardware and special UPS 
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supplies. Figure 5 illustrates a process 500 that uses commonly available UPS 
control lines. Such common control lines enable control of UPS output voltages. 
Other lines enable the UPS to signal systems that a power failure can be anticipated 
(such as when a UPS battery is starting to achieve a high discharge state). Again, 
the overall purpose is to ensure that information regarding which sub-system/power 
supply is powered by which UPS/ AC power line. The process 500 starts at step 
502 and proceeds, at step 504, by having all of the UPS devices produce power. At 
step 506, the sen/ice processor (102 or 152) sends power microcode to a selected 
UPS device that causes that selected UPS device to move its output voltage outside 
of an allowed range (either higher or lower, including OFF). 

[0037] Then, at step 508, the out-of-range condition is sensed. This is commonly 
done by incorporating a circuit in each sub-system/power supply that produces a 
warning signal when a UPS device applies voltage outside of the allowed range. At 
step 510, the sub-system or the service processor determines which power 
supply/server has produced a warning. In practice, an optional verification step, 
step 512, may be useful in ensuring that a warning is not being produced in the 
normal course of events. For example, the service processor might change the 
power microcode to cause the UPS to apply a voltage within the allowed range. If 
the warning is being produced in the normal course of events, the warning will 
remain. If the warning is caused by the power microcode the warning will terminate. 
If the warning is not verified at step 514 the process beneficially loops back to step 
506 for a repeat of the process. 

[0038] However, if the warning is verified, at step 51 6 the service processor 
maps the UPS to the servers/power supplies having warnings. If there are other 
UPS devices to be mapped, at step 518, the process loops back to step 506 for the 
selection of another UPS. However, if there are no other UPS devices to be 
mapped, at step 520, the service processor verifies the UPS integrity. That is, the 
service processor checks that each power supply of a sub-system is powered by a 
different UPS device. Then, at step 522, an optional verification is made to ensure 
that the UPS devices that power each sub-system are not connected to the same 
AC power line. If either step 520 or step 522 fail, signaling a lack of power integrity, 
a warning is provided to the system or to the system operator. Then, at step 524, the 
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process stops. 

[0039] A useful feature of the systems 100 and 150 is the provision for automatic 
shutdown. While the UPS devices 120 and 122 (and 164-168) are powered by 
different AC power lines, it is possible for all power lines to simultaneously fail. In 
that case, warnings can be created by the different UPS devices that signal AC 
power failures can be anticipated. When the service processor (102 or 152) 
determines that a sub-system is in danger of failure the operating software provides 
for a controlled shut down of the sub-system or sub-systems at risk. This can be 
performed in such a way that problems caused by the failure can be prevented or 
mitigated. 

[0040] While the foregoing is directed to embodiments of the present invention, 
other and further embodiments of the invention may be devised without departing 
from the basic scope thereof, and the scope thereof is determined by the claims that 
follow. 
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